System, Method and Computer Program Product for Extracting User Profiles and Habits Based on Speech Recognition and Calling History for Telephone System Advertising

ABSTRACT

A system, method and computer program product for providing targeted messages to a person using telephony services by generating user profile information from telephony data and using the user profile information to retrieve targeted messages.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/681,892, filed Aug. 21, 2017 (now U.S. Pat. No. 10,311,485), which isa continuation of U.S. patent application Ser. No. 14/706,849, filed May7, 2015 (now U.S. Pat. No. 9,741,055), which is a continuation of U.S.patent application Ser. No. 13/768,436, filed Feb. 15, 2013 (now U.S.Pat. No. 9,053,496), which is a continuation of U.S. patent applicationSer. No. 13/300,461, filed Nov. 18, 2011 (now U.S. Pat. No. 8,411,830),which is a continuation of U.S. patent application Ser. No. 11/625,684,filed Jan. 22, 2007 (now U.S. Pat. No. 8,090,082), which claims thebenefit of U.S. Provisional Patent Application No. 60/743,159, filedJan. 23, 2006, each of which is incorporated by reference in itsentirety.

BACKGROUND

Voice communication encompasses a rapidly evolving mix of technologies.A relatively recent communication technology that has garnered a lot ofattention is Voice over Internet Protocol, referred to as VoIP. VoIP canuse the Internet to transmit telephony data (voice and control dataassociated with a VoIP telephone call) in order to provide voiceservices to consumers. The Internet is a publicly accessible worldwidesystem of interconnected computer networks that transmit data using astandardized Internet Protocol (IP) as well as other standard andaccepted data transmission protocols such as Transport Control Protocol(TCP) and User Datagram Protocol (UDP). It is made up of thousands ofsmaller commercial, academic, domestic, and government networks and isused to transmit and host various information and services, such aselectronic mail, online chat, and the interlinked Web pages and otherdocuments on the World Wide Web (WWW).

The main attraction of VoIP technology to businesses and consumers isthe lower cost. VoIP technology has been adopted for use by businessesand consumers as a substitute for existing landline or mobile telephoneservices that use expensive cellular networks and/or the Packet SwitchedTelephone Network (PSTN). By using VoIP, a consumer can make telephonecalls using a broadband Internet connection instead of a regularlandline or mobile telephone. VoIP voice data is transmitted over apacket-switched network by breaking down voice signals into packets ofdigital data from the transmitting end of a telephone call (or acomputer equipped with a microphone), then sending the data over theInternet using UDP (User Datagram Protocol) to the receiving end of thecall. The voice signals are reassembled and played at the handset(telephone) at the receiving end of the telephone call (or through acomputer if it is being used in lieu of a telephone handset to receiveand place VoIP telephone calls). Control data used to initiate a VoIPtelephone call is also transmitted over the Internet.

Because it can utilize existing data communication infrastructure put inplace for Internet data transfer, the overhead for a VoIP serviceprovider is less than that of the more traditional telephone serviceproviders that typically install, maintain, and upgrade their respectivenetworks and communications equipment. As described in more detailbelow, the VoIP service providers do have to provide some equipment tomanage their network of VoIP handsets (telephones) or computers, butsuch equipment is less expensive to install, maintain, and update thanthe network infrastructure supporting the more traditional voicecommunication telephone network (known as PSTN or Public SwitchedTelephone Network). The reason for the lower cost is that the VoIPservice providers can leverage the already existing packet-switchednetwork infrastructure in place to transfer data over the Internet (orover an Intranet for a business). Furthermore, voice communication overPSTN operates over a circuit-switched, rather than packet-switched,protocol. Therefore, an active call requires a 64-kps connection betweenthe parties that cannot be used for any other purpose during the call,and is billed by the service provider accordingly. In contrast, the VoIPpacket-switched approach allows bandwidth that is not being used by thevoice data to be allocated to other purposes.

It would be beneficial to be able to further lower the cost of providingVoIP service to consumers. By lowering the cost of its service, a VoIPservice provider will be able to attract a greater number of consumerscurrently using traditional PSTN and mobile telephone networks. Lowercost is the primary attraction of consumers to VoIP to begin with. So,further reduction in cost will naturally lead to more consumer interest.Additionally, if an individual service provider can provide a lower costservice that is otherwise comparable in quality and features tocompeting VoIP service providers, the more affordable provider willbenefit from customer migration from other VoIP service providers,because the main incentive for such customers to switch to VoIP in thefirst place was to achieve greater cost savings.

One of the most common business models used for generating incomethrough the use of media is advertising. Many businesses successfullyuse advertising revenue as a primary business model through theInternet. Many more Internet-based businesses supplement other revenuestreams through advertising. Traditionally, advertising throughtelephone calls has been limited. It has mostly consisted oftelemarketing calls or prerecorded messages played while a consumer ison hold for some other purpose. Telemarketing advertisements can eitherbe random or targeted but are not typically initiated by the serviceprovider and therefore do not provide a traditional revenue stream toservice providers that can offset the cost of such a service.

However, if a service provider can reliably deliver effectiveadvertisements to its customers it would naturally be able to use aportion of the advertising revenue to offset the cost of providing theservice. So, for example, if an advertisement can be delivered to a VoIPconsumer on that consumer's telephone, for example, as a voice messagepreceding the phone call (there are many other ways to effectivelydeliver such an advertisement, some of which are discussed below), thenthe income generated from the advertisement can be used to offset, oreliminate, the actual cost of the telephone call to the consumer. Thechallenge is to be able to deliver effective targeted advertisements tosuch consumers.

Consumers who would receive such advertisements would understand thatthey are receiving discounted service costs in exchange for having tolisten to or view the advertisement. However, advertisements that haveno relevance to these consumers would be annoying to them, and afterexperiencing frustration with viewing irrelevant advertisements, suchconsumers might instead optionally choose to pay more money for anadvertisement-free service. Therefore, there is a need for a system thatallows the service provider to send targeted advertisements to theseconsumers. Such targeted advertisements would be directed to thoseconsumers' perceived interests or needs and would therefore not beannoying (or as annoying) for the consumers to view or listen to.Furthermore, advertisers would be willing to pay more money if theiradvertisements were being targeted to customers with a particularinterest in their products or services.

Advertising revenue is generated by many on-line businesses. There areeven advertising networks (also known as online advertising networks orad networks) that represent a number of web sites that sell onlineadvertising space, allowing advertisers to reach broad audiencesrelatively easily through a single package deal purchase. Often theseadvertisers pay per click, i.e., they pay a predetermined price forevery click on their advertisement by a web user (such clicks will oftenbring the web user to the advertiser's website). Advertising networksprovide a way for media buyers to coordinate advertising campaignsacross dozens, hundreds, or even thousands of sites in an efficientmanner. The campaigns often involve running advertisements over acategory (run-of-category) or an entire network (run-of-network).

Another online advertising method is called opt-in e-mail advertising(also known as permission marketing), which communicates anadvertisement by e-mail where the recipient of the advertisement hasconsented to receive it. Often the consent is the result of offers offree merchandise in exchange for filling out a survey. Some of theadvantages of this method are that it provides a direct contact with theconsumer and is inexpensive, flexible, and simple to implement. By usingthe information in the survey, in some cases, the advertisements may, tosome extent, be targeted to the consumer's interests. However, unlikethe targeted advertisements achieved through the present disclosure,there is no incentive for the consumer to continue to receive or viewthe advertisements after receiving their free merchandise.

Another method of online advertising is spamming. Spamming is thesending of unsolicited e-mails, usually trying to sell products orservices, to web users. While spamming can be economically viablebecause advertisers have very few operating costs beyond the managementof their mailing lists, it is widely reviled due to the oftenunacceptable content of the e-mails as well as being an annoyingdistraction to e-mail users that do not wish to receive them but have totake time to delete the spam e-mails from their inboxes. That is whyspamming restrictions have been the subject of legislation in a numberof jurisdictions. Spamming also presents a problem because the volume ofunsolicited mail it creates results in costs borne by the Internetservice providers (which is, in turn, indirectly borne by the serviceproviders' customers); the service providers may be forced to add extracapacity to cope with the increase in bandwidth or alternatively providea slower service to its customers.

Contextual advertising is where advertising networks display text-onlyadvertisements that correspond to the keywords of an Internet search orto the content of the page on which the advertisement is shown.Contextual advertisements are believed to have a greater chance ofattracting a user because they are based on the user's search query asthat correlates to the user's interest at the time of query. Contextualadvertising can be seen, for example, in a search query for “wine” whichmay return an advertisement for a wine seller's website.

GOOGLE ADSENSE, for example, implements contextual advertising byproviding its website customer with JavaScript code that, when insertedinto its web pages, generate relevant advertisements from the GOOGLEinventory of advertisers. The relevance of the advertisements shown iscalculated by a separate GOOGLE program that indexes the content of theweb page.

Telephone advertising by telemarketers is accomplished through the useof live sales people or pre-recorded messages. An example of livetelephone advertising is where a sales representative calls a consumeron the telephone to sell products or services. Pre-recorded telephoneadvertising is also used when a caller is put on hold while trying toreach an operator or customer service. For example, a pre-recordedtelephone advertisement about a new model television being offered by anelectronics manufacturer might be played while a caller is put on holdwhile trying to reach technical support.

With the increasing popularity of VoIP, telephone advertising has alsobeen modified and inserted into VoIP calls. For example, a pre-recordedadvertisement may be inserted immediately before a VoIP call isconnected. Further, since many VoIP calls are initiated using acomputer, advertisements may be continuously displayed on the computerscreen while a VoIP call is in session. Alternatively, if the VoIP callis being initiated from a telephone that has a video screen, theadvertisement could be displayed on that screen. However, as discussedabove, such random advertisements may annoy the VoIP consumers and causethem to choose an advertisement-free service.

Speech recognition technologies allow computers to convert wave forms ofhuman speech into text. A typical system for accomplishing speechrecognition consists of a computer equipped with a microphone andspecial speech recognition software. The microphone might also be usedto convert the analog voice wave form into digital data representing thespoken voice so that it can be analyzed and converted to textual form bythe software running on the computer. One well-known speech recognitiontechnique used in such software extracts the sounds that group togetherto form words, referred to as phonemes, from the digital data. Oncethese phonemes have been extracted and recognized, they are convertedinto textual words. A common method of converting these phonemes intowords is through the use of a hidden Markov model (HMM). An HMM is astatistical model that is applied to a set of phonemes to generate themost likely corresponding words.

Speech recognition technologies are often used in transcription. Forexample, speech recognition can be used by people to interact with acomputer who would otherwise have difficulty using a keyboard, such aspeople with physical limitations like carpal tunnel syndrome. Speechrecognition is also used in legal and medical transcription and for thegeneration of subtitles for television programs. Many automatedtelephone services' directory systems also employ speech recognition.For example, there are automated telephone-based directory systems fortravel booking and information, financial account information, customerservice call routing, and directory assistance that utilize speechrecognition technology.

Since under VoIP, the voice data can be (and usually is for VoIPconsumers) transmitted via the Internet, there is no premium forlong-distance or international calling, which is one of the ways thatconsumers benefit from cost savings. By analogy, when an Internet useraccesses a web page in the United Kingdom from the United States, he orshe does not pay any kind of premium international rate but rather onlythe cost incurred for the basic fee from the Internet service provider.Similarly, for example, under VoIP, a telephone call placed from theUnited States to the United Kingdom might have no premium chargesassociated with an international call.

Typically, VoIP voice packets (sometimes referred to as the bearerpackets) are transmitted using UDP over IP. UDP is one of the coreprotocols used in the Internet protocol suite. UDP is used by programsrunning on networked computers to send datagrams (short packets ofinformation) to each other. UDP is more suited to voice communicationdata than other packet-switched data transmission protocols such asTransport Control Protocol (TCP) because UDP is faster and moreefficient, which are very important characteristics for the successfultransmission of real-time voice data. TCP, on the other hand, is bettersuited for reliability because it has built in error checkingfunctionality.

There are several types of VoIP call-control protocols. H.323 is themost widely deployed. Other protocols used include Simple Gate ControlProtocol (SGCP), Internet Protocol Device Control (IPDC), Media GatewayControl Protocol (MGCP), and Session Initiation Protocol (SIP). Some ofthese protocols, such as H.323, were created to deal with real-timemultimedia transmission over an unreliable data network but notspecifically created for VoIP. H.323 is a standard protocol approved bythe International Telecommunication Union (ITU) in 1996 to promote voicetransmission over the Internet and provide mechanisms for voice andvideo communication and data collaboration.

These protocols are interchangeably used in connection with VoIP toaccomplish the same thing, i.e., to accomplish call-flow over thepacket-switched network. Typically, the VoIP service provider willmaintain a call manager that is used to establish the VoIP calls. When aconsumer initiates a call from his VoIP handset (or computer), thecontrol data will be transmitted to the call manager via thepacket-switched network. The call manager will then establish the callby transmitting control data back to the IP address of both theinitiating end and the receiving end of the call, letting them know toinitiate the normal call protocols (such as causing the phone to ring atthe destination end) and transmit the packetized voice (bearer) datadirectly to each other's IP addresses once a call is initiated. So, thecall manager maintains a list of VoIP calls that can translate telephonenumbers into IP addresses as appropriate if both handsets are on theVoIP network.

A gateway typically is established and maintained by the VoIP serviceprovider to make the connection to/from the PSTN network. If, forexample, the VoIP-initiated destination call's destination is in thePSTN network, then the gateway establishes a circuit-switched call onthe PSTN network—which is a circuit-switched rather than packet-switchednetwork that employs time division multiplexing (TDM). In that case,once the call is established the VoIP voice (or bearer) packets will betransmitted to the IP address of the gateway which will be expectingthose packets after receiving the control data from the manager andwill, in turn, transmit the voice data to the destination over the PSTNnetwork using TDM.

Similarly, telephone calls coming from the PSTN network that aredestined to a VoIP pass through a similar gateway that will convert theTDM call data into a packet-switched format for transmission over an IPnetwork. There may also be VoIP gateways internal to the IP network, aswell as DNS servers and other network control devices that need to beset up and maintained by the VoIP service provider, depending on how thenetwork is architected.

The bearer packets are transmitted over the packet-switched networkusing UDP because it is not necessary to try to correct for lost packetson a voice call since the call is happening in real time and any suchcorrection would cause jitters on the call. Therefore, it would beimpractical to use an error-correcting protocol such as TransportControl Protocol (TCP).

Heretofore, there is a need to provide contextual messaging methods andsystems, wherein the particular messages provided to a voice customerare based on keywords extracted from that customer's telephoneconversation using speech recognition technologies or other telephonydata. The messages can include different types of information (e.g.,advertisements, weather, transportation routes, local and foreign news,schedules, historical information, and the like).

SUMMARY

The disclosure provides a system, method, and computer program productfor supplying at least one targeted message to a client device incommunication with the server. An analyzer engine analyzes telephonydata and generates user profile information based on that data. Adatabase stores messages. A search engine queries the database using theuser profile information to retrieve one or more of the messages, and abroadcast engine communicates one or more of the retrieved messages to aclient device.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the present disclosure will become moreapparent from the detailed description set forth below when taken inconjunction with the drawings in which like reference numbers indicateidentical or functionally similar elements. Additionally, the left-mostdigit of a reference number identifies the drawing in which thereference number first appears.

FIG. 1 is a diagram depicting a VoIP system:

FIG. 2 is a diagram depicting a VoIP system enabled to implement oneembodiment of the present disclosure;

FIG. 3 is a diagram depicting a VoIP system enabled to implement asecond embodiment of the present disclosure; and

FIG. 4 is a flow diagram describing steps of a process according to thepresent disclosure.

DETAILED DESCRIPTION

The present disclosure is directed to a system, method, and computerprogram product for creating user profile information from telephonydata, which, in turn, is used to retrieve and display advertisements.

The present disclosure provides a solution to the needs described aboveby allowing such VoIP consumers to receive targeted messages such asadvertisements, weather, bus routes, news, schedules, and the like,which are both less annoying than non-targeted messages because theycorrelate to the interests of the consumer and more desirable to contentproviders such as advertisers, because they are more likely to result ina sale of a product or service.

With the proliferation of packet-based voice services such as VoIP, sucha targeted messaging system as the one described by the presentdisclosure has now become practical. The increase in speed of BroadbandInternet connections has allowed more people to have fast Internetconnections that can be the basis for VoIP services. Also, thecontinuing refinement of speech recognition technologies allows for thattechnology to be utilized according to the present disclosure in orderto generate user profile information that can be used to retrievetargeted messages.

An exemplary embodiment of the present disclosure is a method ofproviding targeted advertisements for consumers by creating user profileinformation based on telephony data using speech recognition and callinghistory to provide the targeted advertising. It should be noted thatthis example is only one possible implementation of the presentdisclosure and is not meant to limit the disclosure to this particularembodiment. After reading the following description, it will be apparentto one skilled in the relevant art how to implement the presentdisclosure in alternative embodiments (e.g., two-way informationdistribution networks such as satellite and wireless terrestrialnetworks utilizing VoIP or other conventional or later-devised voicecommunications service).

It should also be noted that the present disclosure could be utilized todeliver other types of targeted information to consumers, aside fromadvertisements, such as weather reports, news, bus routes, andschedules, points of interest, and the like. For example, if theconsumer is discussing museums and is in the vicinity of a museum asdetermined from the telephony data, one system embodiment of the presentdisclosure might provide the consumer with information about the nearestmuseums and/or directions such as train or bus routes to the museums.

In one embodiment of the present disclosure, the method of providingtargeted advertisements to a consumer using a VoIP telephone serviceincludes the steps of converting the user's speech from a telephone callinto text data through speech recognition software, sending the textdata to a server, extracting keywords from the text data, using thosekeywords to select targeted advertisements stored in a database, andthen providing the selected advertisements to the consumer.

In this embodiment a user can be provided with the most relevant andappropriate advertisements in real time. In this manner, theadvertisements may be specifically chosen to correspond to the subjectof the consumer's conversation. The targeted advertisement may beprovided to the consumer as text, images, audio, and/or video, and insome cases, the advertisement may be provided to the consumer eitherimmediately after the conversation is over, right before the next callmade by the consumer, or while the conversation is taking place. Forexample, if the consumer is connected to a computer, the advertisementmay be displayed in the consumer's Web browser, or if the consumer isusing a mobile phone, the advertisement can be played on the videoscreen of the mobile phone. Optionally, headphones can be connected tothe mobile phone, or a speakerphone feature can be used with the mobilephone to allow easier access to the phone's display. Alternatively, userprofile information can be stored, and advertisements may be provided tothe user at a later time.

Another aspect of the present disclosure encompasses a method thatprovides targeted advertisements through the steps of storing theconsumer's call history, then analyzing the call history to determineuser profile information based on one or more of the following data:date, time, and duration of telephone calls placed and received, name,type, geographical information, Standard Industrial Classification (SIC)code for businesses, if applicable, and reverse lookup information aboutthe callers making or receiving the telephone call at the far end. Themethod may further include the steps of inserting collected user profileinformation into a database, selecting targeted advertisements from adatabase using the user profile information, and providing theadvertisements to the consumer through text, sound, image, and/or video.For example, if a consumer frequently dials the number of a lifeinsurance business, life insurance-related advertisements can beprovided to the consumer.

A number of different available technologies to perform speechrecognition may be used. In one embodiment of the present disclosure,speech recognition is performed using hidden Markov models (HMMs).However, other types of speech recognition technology may be used, someof which are described in “Sphinx-4: A Flexible Open Source Frameworkfor Speech Recognition” (Walker, et al., Sun Microsystems Inc., 2004),which is hereby incorporated by reference in its entirety.

Many ways of performing speech recognition to convert voice data intotext data can be implemented. For example, such conversion could takeplace on either a server or on a client. Additionally, audio streams maybe analyzed either in real-time or on a delayed basis. It may be moreconvenient to convert the audio stream into text data in real-time onthe client side (the point on the network where the consumer isspeaking). If not, then, because it may be the case that the voicepackets in a VoIP call will normally be transmitted directly to therecipient without passing through the server performing the speechrecognition, it might be necessary to also transmit those same datapackets containing the voice data separately to the server performingthe voice recognition. Therefore, under those circumstances, it wouldsave time and bandwidth to convert the voice to text data at the clientside, and then only transmit the converted text data to the advertisingserver. In this scenario, once the text data is received at the server,it can be used to retrieve targeted advertisements which are thentransmitted back to the client. Alternatively, the data packets or thetext data may be saved, cataloged, and/or indexed in a database forlater processing.

The server may also analyze (or provide additional analysis of) the textdata generated from the consumer's conversation. In one embodiment,instead of simply using the text data to directly select the generatedtext data, other data may be processed first to further refine the userprofile information before it is used. One way to analyze the text datais to calculate keyword densities in the converted text, i.e., thenumber of times a word or phrase appears in the text. The more numerouskeywords may be used preferentially to select advertisements more likelyto interest the consumer.

As mentioned above, either in addition to the text data, or instead ofthe generated text data, one or more of the following additional typesof information can be taken or derived from the telephony data arisingout of a telephone call (and preferably a VoIP call): number and partydialed, Standard Industrial Classification (SIC) code and industry ofthe number dialed, existing call history, and the content of the user'sreal-time conversation. The advertisement database can be indexed on anyone or more (up to all) of these data types, in addition to, or apartfrom, having a keyword index.

Using the present disclosure, a VoIP service provider can create arevenue stream from the advertisers supplying the targetedadvertisements. This revenue, in addition to increasing profits for theservice provider, can offset some or even all of the costs associatedwith providing the VoIP services. Customers may, for example, make freelong distance calls in exchange for receiving targeted advertisementsthrough its service. The service provider might also have a noadvertisement option for customers who choose to pay more for the voiceservices in order to avoid the advertisements.

Using an embodiment of the present disclosure that provides forreal-time targeted advertisement, a customer might say “life insurance”during his or her phone conversation, and the system of the disclosure,after converting the phrase to text, could quickly provide a lifeinsurance advertisement to that consumer.

The advertisements may be provided audibly and/or visually and canconstitute one or more of text, sounds, images, and/or video. Theadvertisement may be played before, during, or after the consumer makesa call. If played after the call, the advertisement could be playedimmediately after, at some later time, or right before the next call isplaced by the consumer.

The advertisement may be a co-registration advertisement. For example,when a consumer visits a website and decides to subscribe or join, theperson generally will provide personal information such as name, e-mailaddress, and mailing address. A co-registration advertisement would,optionally, offer that consumer another product or service, such as theopportunity to subscribe to other registration-only web sites withouthaving to retype all of the required information. The following is anillustrative example: after consumer A registers to become a member of awebsite that offers a joke-of-the-day, he or she is offered theopportunity to also subscribe to the cartoon-of-the-day website withouthaving to fill out another registration form.

A co-registration advertisement that might be used according to thepresent disclosure could be an opt-in co-registration advertisement,which is when a person viewing the co-registration advertisement acceptsthe additional offer or offers through an indicating action such as byclicking a checkbox or pressing a button.

Another co-registration advertisement that might be used according tothe present disclosure could be an opt-out co-registrationadvertisement, where the acceptance of the additional offer ispre-checked, and has to be unchecked if the consumer does not want it.

A third type of co-registration advertisement that might be usedaccording to the present disclosure is a custom co-registrationadvertisement. A custom co-registration advertisement may requireadditional information from the consumer specific to the additionalco-registration offer. For example, a custom co-registration offer tosubscribe to a car maintenance website might require that the consumerindicate the type of car he or she owns.

FIG. 1 depicts a diagram showing the components of a VoIP system 100.This diagram shows three potential VoIP consumers using the Internet astheir IP network for packet transmission. One consumer is represented bytelephone 101 attached to a VoIP telephone adapter 102. Normally theVoIP telephone adapter 102 will convert voice traffic from the telephone101 into data packets for transmission over an IP Network 103, which inone embodiment is the Internet. The VoIP telephone adapter 102 iscapable of supporting the protocols used for the VoIP service, e.g.,H.23 or SIP for call control, and UDP/IP for bearer transmission. Inaddition, the VoIP telephone adapter 102 supports call features that areused on the telephone 101, such as, for example, caller identification,call transfer, or three-way calling. Often these call features areprovided for in the call control protocol. The VoIP telephone adapter102 may optionally be combined with the telephone 101.

In another embodiment of the present disclosure, system 100 is a two-wayinformation distribution network such as satellite and wirelessterrestrial networks, which utilizes VoIP or a different protocol forproviding a voice communications service. It should also be understoodthat later-devised services can be used instead of a VoIP service andstill be within the scope of the disclosure.

FIG. 1 also shows another telephone 106 and VoIP telephone adapter 105for a second consumer to place and receive phone calls over network 103.Alternatively, a consumer might utilize a computer 108 (optionally withheadphones/microphone 109) connected to the Internet via a cable modem107 to place and receive VoIP calls.

When the consumer at telephone 101 places a call to the consumer attelephone 106, control data is sent from the VoIP telephone adapter 102to the VoIP manager 104. This control data contains the destination ofthe call, e.g., the telephone number of the VoIP telephone 106. VoIPmanager 104 determines the IP address corresponding to telephone 106,e.g., the IP address of the VoIP adapter 105, initiates the telephonecall by transmitting control data to the respective VoIP adapters,which, in turn, causes the telephone 106 to ring. The VoIP manager 104also communicates the IP address of the respective VoIP telephoneadapters back to each of the VoIP telephone adapters involved in thecall so they can send the voice data directly to each other over thenetwork 103.

Once the call is initiated, data packets containing the voice data aresent from the VoIP telephone adapter 102 directly to the IP address ofVoIP telephone adapter 105, where they are converted back into a voicestream for playback on the telephone 106. The same process occurs in theother direction, with data packets being sent from VoIP telephoneadapter 105 to VoIP telephone adapter 102, where they are converted backinto a voice stream for playback on the telephone 101.

Many calls made from a telephone on a particular VoIP network may not bedestined for telephones on the same network. In such case, a gateway 110(also referred to sometimes as a media gateway) can be used to connectwith other networks. A media gateway controller (“MGC”) within gateway110 handles the registration and management of resources at the mediagateway and exchanges messages with central office switches via asignaling gateway. A media gateway controller is sometimes called a callagent, call controller, or softswitch. Gateway 110 connects to the PSTNnetwork 111. Gateway 110 has an IP address on the Internet known by theVoIP Manager 104 for receiving VoIP voice packet data from the Internetand can also convert a packet data to TDM data, establish a circuit onthe PSTN network 111 and transmit control and voice data using TDM tocomplete the call coming from the VoIP network to send the data to, forexample, a telephone 112 on PSTN 11. The gateway 110 also can initiate acall on the VoIP network 100 that comes, for example, from the PSTNnetwork 111, by sending control data to the VoIP manager 104 andreceiving the IP address of a destination VoIP telephone adapter fromthe VoIP manager 104. Gateway 110 converts the TDM voice data from thePSTN network 111 into packets and transmits that packet data directly tothe IP address of the destination VoIP telephone adapter.

FIG. 2 is a diagram depicting a VoIP system 200 enabled to implement oneembodiment of the present disclosure. In one embodiment of the presentdisclosure, client software 216 resides in, or alongside, the VoIP phoneadapter 205. When a call is placed, for example, from telephone 206 totelephone 201 through network 203 and VoIP adapter 202, the VoIP adapter205 will send the control data over the network 203 to the VoIP manager204 to initiate the call. A call could also be placed from a computer208, using headphones 209 and a cable modem 207. The control data, orsome extracted part of the control data is also sent to a server 215.Server 215 can be apart from or integrated into the VoIP manager 204, ora combination of both. The control data is used as user profileinformation to retrieve data including, for example, advertisementinformation from the advertisement database 214, which in turn istransmitted back to the VoIP phone adapter 205 for output on thetelephone 206. Output of data can constitute playback of audio, and/orthe display of text, graphics, images, and videos, and the like. Theretrieved data also need not be limited to advertisement data and can,for instance, include other information (e.g., weather forecastinformation, public transportation information, local news, to name afew). In addition, or alternatively, the user profile information can besaved in a user profile database 213.

In another embodiment, the client software 216 optionally also includesvoice recognition software which analyzes the voice data, either inpacket form from the VoIP phone adapter 205, or directly from thetelephone 206. The voice recognition software converts this voice datainto text data. The text data is then transmitted over the network 203to server 215 where it can either be stored in a user profile database213 or used to select targeted advertisements from the advertisementdatabase 214, which are then transmitted back to the VoIP phone adapter205 for display to the consumer using the telephone set 206.

Optionally, the user profile database 213 could be stored in the devicehosting the VoIP phone adapter 205 local to the telephone 206 instead ofon the server 215. In this manner, one server 215 can host many clientswithout adding additional identifying information for each client to theserver's database since the user profile information is distributed foreach client. It should be noted that, according to the presentdisclosure, the targeted advertisements do not have to be provided tothe consumer in real time. Since the user profile information can bestored in a user profile database 213, it can be used to later selectadvertisements from the advertising database 214, or the user profileinformation can be further processed to refine the advertisementselection criteria.

FIG. 3 shows another embodiment of the present disclosure where thevoice recognition software 316 runs on the server 315 rather than aclient, such as VoIP adapter 305. Similar to FIG. 2, a call can beinitiated by VoIP-configured telephone 309, by the VoIP adapter 305 atthe initiation end of the call transmitting control data to a VoIP callmanager 304, through the network 303. The VoIP call manager 304 willsend control data to the VoIP adapter 302 at the destination end of thecall, which, in turn, will cause the telephone 301 to ring. In oneconfiguration, the voice recognition software 316 runs its conversion ofvoice data into text data by having the voice data locally available onthe server 315. Under that scenario, the voice packets, or bearer data,are transmitted to the server 315 for analysis by the voice recognitionsoftware 316. Another possible configuration is to have the voicerecognition software launched off of the server 315 but remotelyaccessing the bearer voice data at the client (e.g., VoIP adapter 305).This can be accomplished in at least two ways. The voice recognitionsoftware 316 can be launched from the server 315 but run locally on theclient once launched. Alternatively, the voice recognition software 316,runs on the server 315 but remotely accesses the voice bearer dataresiding on the client through a data pipe (not shown).

Once the user profile information is generated, it can be stored in theuser profile database 313 and/or used to select targeted advertisementsfrom the advertising database 314. Once selected, targetedadvertisements are transmitted back to the VoIP adapter 305 and thencommunicated to the consumer (e.g., the data is output by playback ofaudio and/or the display of text, graphics, images, and videos, and thelike). It is also possible, as an alternative, to store the user profileinformation locally on the client in this configuration.

Referring to FIG. 4, a flowchart illustrating the process of providing atargeted advertisement to a user, according to one embodiment of thepresent disclosure, is shown. The process is divided into two sides, theclient side 421 and the server side 422. A user obtains or is providedwith the client software application that implements functionalityaccording to the present disclosure on the client side. One exemplaryway of obtaining the software application is to download it from aremote website. Another exemplary way of obtaining the softwareapplication is from a computer-readable medium provided in the form of aCDROM, DVD, memory device, and the like.

The user places a VoIP call as shown in step 424. This process may takeplace in a computer or through a telephone. Particularly, a user entersa telephone number to be called by selecting physical buttons on atelephone, selecting numbers on a dialpad displayed on a webpage, or bysaying the number into a microphone which, in turn, are voicerecognized, to name a few.

After initiating a call in step 424, control data is transmitted to theVoIP call manager 204 in step 425 which, in turn, sends control data toboth the origination end and destination end of the call which isreceived at the client (e.g., VoIP adapter 205) in step 426 to establishthe connection. Once the connection is established, the voice datapackets will be transmitted directly back and forth over thepacket-switched network using the UDP protocol in step 427. The voicedata are converted to text strings using voice recognition software instep 428. The text strings are transmitted to the server side 422 instep 429, and received by the server in step 430. On the server side,keywords are extracted from the text strings in step 431. At this point,the server might optionally store those keywords as user profileinformation for the particular user that initiated the call. Thekeywords are then used to select targeted information (e.g.,advertisements) from a database such as advertisement database 214 instep 432. The targeted information is transmitted back to the client insteps 434 and 435. Finally, the targeted advertisements are output on aclient device to the user in step 436. The process is then repeated ifthe call is still in progress, going back to the step 427 whereadditional voice data is generated for transmission to the VoIPdestination IP address which is then converted into text strings usingvoice recognition software.

Alternatively, although not shown in the figure, the conversion of thevoice data to text strings in step 428 may be carried out in the serverside 422. In such alternative embodiment, the voice data packets may betransmitted to the server. Once at the server side 422, a copy of thevoice data may be saved in a database or as a file on the server side422. The voice data may also be converted to text strings using a speechrecognition software on the server side 422.

The process of keyword extraction 431 may also be performed on theclient side 421. In any event, the keywords may then be saved into adatabase (not shown in the diagram). The database contains a pluralityof advertisements. The advertisement database may associate keywordswith specific advertisements; so when keywords are extracted from thevoice data through speech recognition software, they are used to selectadvertisements from the advertisement database by searching for theadvertisements' associated keywords. For example, an advertisement on atravel package may have the following keywords associated therewith:travel, package, flight, airplane, vacation.

In step 432 the advertisements are selected based on keywords that wereconverted from the voice data. Once it has been determined that acorresponding advertisement exists in the database, the information ofthe corresponding advertisement is retrieved, as shown in step 432. Theretrieved advertisement may be sent to the user to be displayed on theuser's computer display, as shown in steps 434-435.

In the alternative, subsequent to the user placing a call in step 424,the number that the user enters on the telephone may be inserted into adatabase for use as a keyword. Optionally, the number can be used to doa reverse lookup for additional information that may also be used as akeyword to select targeted advertisements. For example, the reverselookup might find the type of business of the dialed number (travelagency, life insurance, hardware store, etc.), or the StandardIndustrial Classification (SIC) code of the business. Thus, for example,if the user dials a travel agency number, it could be determined throughthe reverse lookup that the dialed number is a travel agency, then thesekeywords “travel” and “agency” or the phrase “travel agency” might beused as the user profile information to select the targetedadvertisements.

In one embodiment, keyword densities may be calculated from theconverted text data. The algorithm used for such a calculation may besimple, e.g., the number of times a word is spoken in a conversation istracked and that number is stored along with the word in the userprofiles' information for that consumer. When the keywords are used toselect the targeted advertisements in step 432, the word with thehighest density number (the word spoken the most times) could be usedfirst to select advertisements. If no advertisements are found, then thenext most frequent word can be used to select at least one advertisementfrom the advertisement database for display to the consumer.

In another embodiment, call history may be stored on the server in auser profile database. The consumer's dialing habits and other callinformation can be collected based on, for example, the date, time, andduration of calls placed and received, party information. e.g., name,type (business, residential, other), geographical information, SIC codefor businesses, and reverse lookup number information. Such informationmay optionally be used as keywords or converted into keywords bysuitable algorithms that analyze such data. Those keywords can then beused to select targeted advertisements from an advertisement database.

The VoIP service provider may set up its VoIP service using SessionInitiation Protocol (SIP) as the control protocol. Alternatively, theapplication and/or the server may use H.323 as the control protocol.

The server may store the advertisement database including a plurality ofadvertisements with associated keywords. There might be a one-to-manyrelationship between advertisements and keywords. There may also be apriority system amongst keywords, where for example, the database schemacould be setup so that certain keywords will bring back severaldifferent advertisements, but the advertisements will be selected intheir order of highest priority for the keywords, so the most relevantadvertisement according to the priority number might be sent to theconsumer for display first, followed by the highest priorityadvertisement, and so on.

As described above, information associated with the keywords (e.g., theadvertisements) can be displayed to the consumer by being played audiblyand/or shown visually through text, images, or videos. The display maycome out on the consumer's computer, or directly on the phone, or othermeans of communication.

A general purpose computer connected to a packet-switched network issufficient for performing the operation on the client side according tothe present disclosure. It might also be desirable for the consumer tohave a handset (telephone) on the client-side. Alternatively, a handsetmay be used with a phone adapter computing device instead of a generalpurpose computer to avoid having to use the computer to initiate atelephone call.

If there is no handset used in conjunction with a computer, then thecomputer can be configured with peripherals to make a phone call, suchas a microphone and speaker. A headset is one possible peripheral thatcould be used to accomplish this according to the present disclosure.The computer system can include a display that shows graphics, text, andvideo for displaying the targeted advertisements.

The computer would also need to have a communications interface, such asa landline or wireless network card, or other means of two-way networkcommunications adapter. A network card allows the computer to connect toa packet-switched network, which will be necessary to transmit voice,text, and control data over the network, to establish and maintain thetelephone call as well as to transmit user profile information, textstrings, voice data and targeted advertisements. Examples of thecommunications interface may include a modem, a network interface (suchas an Ethernet card), a communications port, a Personal Computer MemoryCard International Association (PCMCIA) slot and card, etc.

While various embodiments of the present disclosure have been describedabove, it should be understood that they have been presented by way ofexample, and not limitation. It will be apparent to persons skilled inthe relevant arts that various changes in form and detail can be madetherein without departing from the spirit and scope of the presentdisclosure. Thus, the invention should not be limited by any of theabove-described exemplary embodiments, but should be defined only inaccordance with the following claims and their equivalents.

In addition, it should be understood that the figures illustrated in theattachments, which highlight the functionality and advantages of thepresent disclosure, are presented for example purposes only. Thearchitecture according to the present disclosure is sufficientlyflexible and configurable such that it may be utilized (and navigated)in ways other than that shown in the accompanying figures.

Further, the purpose of the foregoing specification and abstract are toenable the U.S. Patent and Trademark Office and the public generally,and especially the scientists, engineers, and practitioners in the artwho are not familiar with patent or legal terms or phraseology, todetermine quickly from a cursory inspection the nature and essence ofthe technical disclosure of the application. The specification andabstract are not intended to be limiting as to the scope of theinvention in any way. It is also to be understood that the steps andprocesses recited in the claims need not be performed in the orderpresented.

1-24. (canceled)
 25. A method comprising: capturing an audio signal;converting at least a portion of the captured audio signal into textdata; transmitting the text data to a remote server; and displaying atargeted message received at the mobile device from the remote server,wherein the targeted message has been selected from a databasecomprising a plurality of messages, based at least in part on the textdata.
 26. The method of claim 25, wherein the targeted message includesat least one of sound, text, and image data.
 27. The method of claim 25,wherein the targeted message has been selected further based on keywordsextracted from the text data.
 28. A method comprising: logging callhistory information for a user of a mobile device, wherein the callhistory information is based at least in part of a plurality of callsinvolving the user; transmitting the logged call history information toa remote server; and displaying a targeted message received at themobile device from the remote server, wherein the targeted message hasbeen selected from a database comprising a plurality of messages, basedat least in part on the logged call history information.
 29. The methodof claim 28, wherein the targeted message includes at least one ofsound, text, and image data.
 30. The method of claim 28, wherein thecall history comprises date, time, and duration of telephone callsplaced and received.
 31. The method of claim 28, wherein the callhistory comprises one or more of name information, call typeinformation, geographical information, business information, or reverselookup information.
 32. A non-transitory computer-readable medium havinginstructions stored therein that, in response to execution by a mobilecomputing device, cause the mobile computing device to: capture an audiosignal; convert at least a portion of the captured audio signal intotext data; transmit the text data to a remote server; and display atargeted message received at the mobile device from the remote server,wherein the targeted message has been selected from a databasecomprising a plurality of messages, based at least in part on the textdata.
 33. The non-transitory computer-readable medium of claim 32,wherein the targeted message includes at least one of sound, text, andimage data.
 34. The non-transitory computer-readable medium of claim 32,wherein the targeted message has been selected further based on keywordsextracted from the text data.
 35. A non-transitory computer-readablemedium having instructions stored therein that, in response to executionby a mobile computing device, cause the mobile computing device to: logcall history information for a user of a mobile device, wherein the callhistory information is based at least in part of a plurality of callsinvolving the user; transmit the logged call history information, loggedcall history information to a remote server; and display a targetedmessage received at the mobile device from the remote server, whereinthe targeted message has been selected from a database comprising aplurality of messages, based at least in part on the logged call historyinformation.
 36. The non-transitory computer-readable medium of claim35, wherein the targeted message includes at least one of sound, text,and image data.
 37. The non-transitory computer-readable medium of claim35, wherein the call history comprises date, time, and duration oftelephone calls placed and received.
 38. The non-transitorycomputer-readable medium of claim 35, wherein the call history comprisesone or more of name information, call type information, geographicalinformation, business information, or reverse lookup information.